Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Arcade layout #31560

Merged
merged 3 commits into from
Dec 10, 2018
Merged

Arcade layout #31560

merged 3 commits into from
Dec 10, 2018

Conversation

tmat
Copy link
Member

@tmat tmat commented Dec 5, 2018

Build output (/artifacts)

The layout of build output is documented in Arcade docs and is the same for all repositories on Arcade SDK. Some directories are optional, e.g. repositories that do not build VS components naturally do not create VSSetup and VSSetup.obj directories.
Repositories may put additional content under VSSetup/{config}, beyond what’s documented in the Arcade SDK.
All builds outputs go under artifacts directory in the repository root.

VSSetup directory

Roslyn adds the following to VSSetup:

  • DevDivPackages: packages inserted to VS CoreXT package feed and other files inserted to VS enlistment
  • Insertion\bootstrapper, Insertion\BranchInfo, Insertion\OptProf – files needed for training of partial NGEN optimization (aka OptProf, aka IBC). This will likely change soon as our OptProf infrastructure is still evolving.
  • Installer: The installer zip (Roslyn_Preview.zip) goes here
  • VSIXes that are inserted to VS are placed under Insertion directory, those that are not inserted to VS are placed directly under VSSetup, e.g. RoslynDeployment.vsix.
  • Projects that build shipping VS components produce VSIXes of 2 flavors:
    o “Experimental”: placed to VSSetup/{config} directory
    o “SystemComponent”: placed to VSSetup/{config}/Insertion

VSSetup directory is published by official build to Azure Pipelines Artifact Store and picked up from there by Roslyn Insertion Tool. Everything needed for insertion must be places to this directory.

Engineering directory (/eng)

This directory contains scripts, targets and configuration used by the build.
The most interesting files for a Roslyn developer are:

  • cibuild.cmd/.sh
    Runs CI build (both PR validation and official builds).
  • test-build-correctness.cmd
    Validates consistency and correctness of the build.
  • test-determinism.cmd
    Validates that our build is deterministic.
  • generate-compiler-code.cmd
    Regenerates generated files used in the compilers (syntax trees, etc.).

You can run the above scripts directly from your local clone. No arguments are required, although you might want to specify configuration if Debug is not what you want to test. The scripts are designed to execute exactly the same logic as on CI machine, modulo real signing and publishing. These run in validation-only mode when executed in non-official build (locally or during PR validation).

Common engineering directory (/eng/common)

This directory contains bootstrapper scripts that are common across all Arcade repositories. The Arcade engineering system will overwrite these periodically, so don’t make any permanent changes here, they will disappear without notice.

Scripts directory (/scripts)

This directory is used for helper scripts that are manually executed and not essential for CI build.

.dotnet, .tools, .packages

These directories might be created in the repository root under certain circumstances. The bootstrapper script uses them to store downloaded .NET Core (when the required version is not found on PATH), vswhere (when building from environment that is not initialized with VS env variables) and restored NuGet packages (on CI when not using global nuget cache), respectively.

Official build artifacts publishing

The official build does not publish to cpvsbuild share anymore. Instead it publishes

  1. VS Setup Components
    These are VSIX packages that are used by VS installer built to artifacts/VSSetup/{config}/Insertion directory
  2. CoreXT packages
    Packages used by VS build – mapped to ExternalAPIs and Tools in Razzle
  3. Artifact VSSetup
    Entire artifacts/VSSetup/{config} directory.
  4. Artifact Packages
    Entire artifacts/packages/{config} directory

Fixes #23547

@tmat tmat requested review from a team as code owners December 5, 2018 22:39
@tmat tmat mentioned this pull request Dec 5, 2018
eng/build.ps1 Outdated

if ($launch) {
$devenvExe = Join-Path $env:VSINSTALLDIR 'Common7\IDE\devenv.exe'
&$devenvExe /rootSuffix RoslynDev
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

&$devenvExe /rootSuffix RoslynDev [](start = 7, length = 34)

Should use Exec-Process here as the exit code isn't validated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure why we have it but I'll update it to use Exec-Process

@jaredpar
Copy link
Member

jaredpar commented Dec 6, 2018

tar czf mono-${VERSION}.tar.gz mono

Why do we still have a scripts directory at all after this change?


Refers to: scripts/build_mono.sh:16 in 54a1ef2. [](commit_id = 54a1ef274b37ea1c9aa3499493eecb240c7d095e, deletion_comment = False)

Copy link
Member

@jaredpar jaredpar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@tmat
Copy link
Member Author

tmat commented Dec 6, 2018

Why do we still have a scripts directory at all after this change?

I moved all scripts that are not used in build/CI there. These can definitely use some clean up.

PublicApi
UploadAzureZip
GenerateSdkPackages
build_mono.sh
check-for-loc-changes.cmd
check-for-loc-changes.ps1
cleanup_perf.ps1
clean_benchview_tools.cmd
crossgen.sh
edit-designers.ps1
generate-badges.ps1
generate-compiler-code.cmd
generate-compiler-code.ps1
install_benchview_tools.cmd
README.md
Run-TestsWithProcessWatchdog.ps1
run_perf.ps1
sort-unshipped.cmd
test-nuget-cache.ps1

@tmat tmat force-pushed the Layout branch 4 times, most recently from 89881e1 to 0913d4e Compare December 8, 2018 18:48
@tmat tmat changed the title WIP: Arcade layout Arcade layout Dec 10, 2018
@tmat tmat merged commit d7e4939 into dotnet:master Dec 10, 2018
@tmat tmat deleted the Layout branch December 10, 2018 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants